The main challenges that arise when adopting Gaussian Process priors inprobabilistic modeling are how to carry out exact Bayesian inference and how toaccount for uncertainty on model parameters when making model-based predictionson out-of-sample data. Using probit regression as an illustrative workingexample, this paper presents a general and effective methodology based on thepseudo-marginal approach to Markov chain Monte Carlo that efficiently addressesboth of these issues. The results presented in this paper show improvementsover existing sampling methods to simulate from the posterior distribution overthe parameters defining the covariance function of the Gaussian Process prior.This is particularly important as it offers a powerful tool to carry out fullBayesian inference of Gaussian Process based hierarchic statistical models ingeneral. The results also demonstrate that Monte Carlo based integration of allmodel parameters is actually feasible in this class of models providing asuperior quantification of uncertainty in predictions. Extensive comparisonswith respect to state-of-the-art probabilistic classifiers confirm thisassertion.
展开▼
